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Abstract 



This paper is a survey on invariants of representations of quivers and their generalizations 
such as (super)mixed representations by Zubkov, orthogonal and symplectic representations 
by Derksen and Weyman, and representations of signed quivers by Shmelkin. Working over a 
field of arbitrary characteristic we present the description of generating systems for invariants 
and relations between generators. 
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1 Introduction 

We work over an infinite field F of arbitrary characteristic. All vector spaces are supposed to be 
finite dimensional and all algebras are associative with unity. 

The paper can be split into three parts. The first part is Section [3] and it is dedicated to 
matrix invariants of classical linear groups. In Section 13.11 systems of generators for algebras of 
matrix invariants are given. Some connections between matrix invariants and representations of 
free algebras are considered in Sections 13.21 and 13.31 Relations between generators for the cases of 
GL(n) and O(n) are described in Section [3. 51 The description in the case of 0(n) is given in terms 
of the polynomial cr tjr , which is defined in Section [3.41 

The second part is Section 2] and it is dedicated to representations of quivers. A quiver is 
a finite oriented graph. A representation of dimension (nj, . . . ,n{) of a quiver with I vertices 
consists of a collection of column vector spaces ¥ ni , . . . , ¥ ni , assigned to the vertices, and linear 
mappings between the vector spaces "along" the arrows (see Section |4~T1 for details). This notion 
was introduced by Gabriel in pjj] as an effective mean for description of some problems of the 
linear algebra. The importance of this notion from the representation theory point of view is 
due to the following fact. Let A be a finite dimensional basic algebra over algebraically closed 
field. Then the category of finite dimensional modules over A is a full subcategory of the category 
of representations of some quiver (see Chapter 3 from |16j). Invariants of quivers are important 
not only in the invariant theory but also in the representational theory of quivers because these 
invariants distinguish semi-simple representations of a quiver. A system of generators for invariants 
of a quiver is given in Section 14.21 and relations between generators are described in Section 14.31 

The third part consists of Sections [5l [6j and [7] and it is dedicated to generalizations of repre- 
sentations of quivers. Given a representation of dimension (nx, . . . , ni) of a quiver with / vertices, 
we generalize this notion as follows. Let v be a vertex of the quiver, where 1 < v < I. In the 
classical case GL(n v ) acts on F* 1 " but in our case an arbitrary classical linear group from the list 
GL(n v ), 0(n v ), Sp(n v ), SL(n v ), SO(n v ) can act on F™". Moreover, we can consider the dual 
space (F"")* together with F™" in order to deal with bilinear forms together with linear mappings. 
Finally, instead of arbitrary linear mappings "along" arrows we can consider only those that, for 
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example, preserve some bilinear symmetric form on "vertex" spaces, etc. Representations of quiv- 
ers obtained in this way are called H-mixed representations, where a mixed quiver setting is the 
data that determines what linear mappings we may take to form a mixed representation, i.e., Q is 
a quiver together with a dimension vector, a product G of classical linear groups, kinds of linear 
mappings along arrows and an involution that shows which vertices are dual. The exact definitions 
together with examples are given in Section [5TTJ 

Particular cases of the considered construction are 

• mixed and supermixed representations of quivers, introduced by Zubkov in [40] (see also [43]); 
to obtain mixed (supermixed, respectively) representations we should take Q-mixed repre- 
sentations such that G is equal to a product of the general linear groups (the general linear 
groups, orthogonal and symplectic groups, respectively); 

• orthogonal and symplectic representations of symmetric quivers, introduced by Derksen and 
Weyman in [5]; 

• representations of signed quivers, introduced by Shmelkin in [34] ; 



(see Example 15.21 for details). The motivation for these generalizations of quivers from the point 
of view of the representational theory of algebraic groups was given in [5] , |34j , where symmetric 
and signed quivers, respectively, of tame and finite type were classified. 

Let us depict different kinds of algebras of invariants and semi-invariants. Here a vertical arrow 
means that the upper class of algebras contains the lower class of algebras. All necessary definitions 
can be found in Sections 13 - H 14. H and 15.11 (see also Examples 15.21 and [ 
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In Section 15.21 generators for invariants of supermixed representations of quivers are given. 
Relations between generators for mixed representations of quivers are described in Section 15.31 
Earlier introduced polynomial at.r plays a key role here. 

To describe generators for invariants of O-mixed representations, we introduce a block partial 
linearization of the pfaffian (b.p.l.p.) and a tableau with substitution (see Section^. Note that the 
polynomial at,r can be defined in terms of a b.p.l.p. (see part 4 of Example 16.81 and Remark l3.14p . 

In Section[7]we present generators for invariants of Q-mixed representations. Before formulating 
the main result in Section [7. 21 we consider a partial case of semi- invariants for the usual represen- 
tations of bipartite quivers. Note that Theorem 17.71 implies the description of semi-invariants of 
supermixed representations of quivers. 

So the main results formulated in this paper are the descriptions of 

• generators for invariants of 0-mixed representations from Section 17.21 

• relations for invariants of mixed representations of quivers from Section 15.31 

• relations for matrix 0(n)-invariants from Section [3.51 

The rest of statements from this paper that concern generators or relations are partial cases of the 
mentioned ones. 



2 Notations 

2.1 Matrices 

Given a positive integer n, let us fix the following notations for the classical linear groups: 

• 0(n) = {A e F nx ™ | AA T = A T A = E} and SO(n) = {A e 0(n) | det(A) = 1}, where we 
assume that the characteristic of F is not two; 

• Sp(n) — {A e F™ x " | A T JA = J}, where we assume that n is even. 



for the the matrix of the skew-symmetric bilinear form on ¥ 2n . We also fix the following notations 
for certain subspaces of F nxn : 

• S + (n) = {A G F nXTl | A T = A} is the space of symmetric matrices; 

• S~(n) = {A G F™ xn | A T = —A} is the space of skew-symmetric matrices; 

• L + (n) = {A e F nxn | A J is a symmetric matrix}; 

• L~(n) = {A e F" x ™ | A J is a skew-symmetric matrix}. 



Denote coefficients in the characteristic polynomial of an n x n matrix X by cr t (X), i.e., 
det(A£ - X) = A™ - a^X)^ 1 + ■■■ + (-l) n a n (X). 
So, (7i(X) = tr(X) and a n (X) = det(X). 



Here E = E(n) stands for the identity matrix, and J — J(n) — 
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Assume n is even. Define the generalized pfaffian of an arbitrary n x n matrix X — (xij ) by 

p7(X)=pf(A-A T ), 

where pf stands for the pfaffian of a skew-symmetric matrix. By abuse of notation we will refer to 
pf as the pfaffian. For F = Q there is a more convenient formula 

n/2 

pfpO =pi(X -X T ) = — — - sgn( 7 r)]Ja; 7r( 2 i _i) >7r (2i). (1) 

^ ' ireS„ i=l 

For n x n matrices X\ — (xij(l)), . . . , X s = (xjj(s)) and positive integers n,. .. ,r s , satisfying 
r\ + ■ ■ ■ + r s = n/2, consider the polynomial pf (x\X\ + ■ ■ ■ + x s X s ) in the variables xi, . . . , x s . The 
partial linearization 

R , V.: 

of the pfaffian is the coefficient at x^ 1 ■ ■ ■ x r s s in this polynomial. In other words, for F = Q we have 

j s riH hr 3 - 

Pf ri r.(- y ivJ») = - ^]sgn(7r)]] JJ aJ w ( 2i _ 1))W ( 2i )(j), (2) 

' ^e5„ j=l i=nH hrj-i + l 

where c = n! ■ • • r s \. The partial linearization det ri (X\, . . . , X s ) of the determinant is defined 
analogously, where X\,...,X S are n x n matrices, r\ + ■ ■ ■ + r s = n, and n is arbitrary. 

2.2 Rings 

In what follows, N stands for the set of non-negative integers. 

Denote by A — ¥[xi, . . . , Xd] the polynomial ring in x\, . . . , Xd over F, i.e., A is a commutative 
F-algebra with unity generated by algebraically independent elements x\, . . . ,Xd- 

Let {fi | i £ /} be a finite or countable system of generators for a commutative algebra A. Then 

A ~ F[a;i | i G /]/T, 

where T is the ideal of relations. 

2.3 Quivers 

A quiver Q — (Q , Qi) is a finite oriented graph, where Q is the set of vertices and Qi is the set 
of arrows. Multiple arrows and loops in Q are allowed. For an arrow a, denote by a' its head and 
by a" its tail, i.e., 

We say that a — a\ ■ ■ ■ a s is a path in Q (where a\, . . . , a s E Qi), if a'{ = a' 2 , . . . , a"_i = ct' s , i.e., 

ai a s 



The head of the path a is a' — and the tail is a" = a". A path a is called closed if a' = a". 
A closed path a is called incident to a vertex v e Q Q if a' = v. Similarly, closed paths 0i, . . . , f3 s 
in Q are called incident to u if /?( = ■•• = (3' s = v. A closed path a is called primitive if a is not 
equal to a power of a shorter closed path, i.e., a ^ (3 l for any I > 1 and any closed path (3 in Q. 
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3 Matrix invariants and representations of free algebras 

3.1 Generators for matrix invariants 

Let G be a group from the list GL(n), 0(n), Sp(n), SO(n) and let 

H = ¥ nxn © • • • © F" x " 

be (i-tuple ofnxn matrices over F. The group G acts on H by the diagonal conjugation, i.e., 

g-(A u ...,Ad) = (gA^^.^gAag- 1 ), (3) 

where g G G and A 1 ,...,A d e ¥ nxn . 

The coordinate ring of H (i.e. the ring of polynomial functions / : H — > F) is the ring of 
polynomials 

¥[H] = ¥[xij(k) 1 1 < i,j < n, 1 < k < d], 

where Xij(k) stands for the coordinate function on H that takes a representation [A\, . . . , Ad) G H 
to the (i, j) th entry of the matrix A}.. Denote by 

/ xn(fc) • • • x ln (k) \ 
X k = : : 

\ x nl{k) •■■ x nn (k) J 

the k generic matrix (1 < k < n). 

The action of G on H induces the action on ¥[H] as follows: (g ■ f)(h) = fig' 1 ■ h) for all 
g G G, f G F[if], h £ H. In other words, 

g-Xij(k) = (i,j) th entry of g~ 1 X k g. 

The algebra of matrix invariants is 

F[ff] G = {/ G F[fl] \g-f = f for all 3 £ G}. 

We do not consider the case G = SL(n) because invariants for GL(n) and SL{n) are the same. 

Theorem 3.1. The algebra of matrix invariants ¥[H] G is generated by the following elements: 

a) o- t (A) (1 < t < n and A ranges over all monomials in X\, . . . , Xd), if G — GL(n); 

b) a t (B) (l<t<n),ifG = 0{n); 

c) <J t {B) (1 <t <n), if G — SO(n) and n is odd; 

d) <Jt{B), pf ri ,...,r s (-^li ■ • ■ > Bs) A — * — n > r i + ' ' ' + r s = n /2j, if G = SO(n) and n is even. 
In b), c), and d) matrices B,B\, . . . ,B S range over all monomials in Xi, . . . , Xd, X\ ', . . . , Xj. 

e) o- t (C) (1 < t < n and C ranges over all monomials in X\, . . . ,Xd, JXfj, . . . , JXjj), if 
G = Sp{n). 



Example 3.2. Let G — GL(n), n = 2. Then a minimal (i.e. irreducible) system of generators for 
the algebra F[7?] G is the following one: 

• tr(X fcl ■■■X kp ), detpf fe ) (1 < p < 3, 1 < fa < ■ ■ ■ < k p < d and 1 < k < d), if the 
characteristic of F is not two; 

• trpffej • • ■ Xk p ), det(Xk) (1 < k% < ■ ■ ■ < k p < d, and 1 < k < d), if the characteristic of F 
is two. 

In particular, the maximal degree of elements from any generating system is equal to 3 (d, respec- 
tively) in the first (the second, respectively) case. If we also have d — 2, then in both cases ¥[H] G 
is the polynomial ring in 

tr(Xx), tr(X 2 ), tv{X 1 X 2 ), det(Xt), det(A 2 ). 



Remark 3.3. In the case of a characteristic zero field it is enough to take traces instead of at, 
1 < t < n, in the formulation of Theorem 13. II 

Remark 3.4. The system of generators from Theorem 13.11 is infinite but it can be shown that 
there exists a finite generating system. In this remark we give references to some results concerning 
finite generating systems. 

From now on we assume that G = GL(n) and consider the algebra of matrix invariants. An 
upper bound on degrees of elements of a minimal system of generators (m.s.g.) was given in [5] 
in terms of the nilpotency degree of a (unitary) relatively free finitely generated algebra with the 
identity x n = 0. 

For small n the algebra of matrix invariants can be described more explicitly. A m.s.g. is known 
for n = 2 (see [35], [32], [S]) and n — 3 (see [23], [35]). In characteristic zero case a m.s.g. for n = 4 
and d = 2 was described in [T5J . 

By the Noether normalization the algebra of matrix invariants contains a system of parameters, 
i.e., a set of algebraically independent elements generating a subalgebra such that the algebra of 
matrix invariants is integral over this subalgebra. A system of parameters for matrix invariants 
was constructed for n — 2 and any d (see [37] and [S]), n = 3, 4 and d = 2 (see [37]), and n = d = 3 
(see [13J). 

For more detailed introduction to finite generating systems for matrix invariants see 
overviews [T7J and [TB] by Formanek. For recent developments in characteristic zero see [T3] and 
in positive characteristic see [9]. 

Remark 3.5. For a field of characteristic zero generators for matrix invariants of G G 
{GL(n),0(n), Sp(n)} were described by Sibirskii in [35] and Procesi in [3T]. Developing ideas 
from [3T], Aslaksen et al. calculated generators for G = SO(n) (see [2]). 

The importance of characteristic-free approach to matrix invariants was pointed out by For- 
manek in overview [18] (see also [17]). Relying on the theory of modules with good nitrations 
(see [H]), Donkin described generators for matrix GL(n)-invariants in [12]. As regards the rest 
of classical linear groups over a field of positive characteristic, the first results were obtained by 
Zubkov. In [IT] he described generators for matrix 0(n)- and ,Sp(n)-invariants. The proof is based 
on ideas from [12) and a reduction to invariants of Q-mixed representations with g v = GL for 
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every vertex v (see below Section [5] for the definition). The reduction was performed by means of 
Frobenius reciprocity. Generators for matrix SO (n)-invariants were described by Lopatin in [28] . 

Remark 3.6. Note that it is possible to define 0(n) and SO(n) in characteristic two case. But in 
this case even generators for invariants of several vectors are not known (for the latest developments 
see [10]). 

3.2 Representations of free algebras 

Consider a free associative algebra 

A = ¥(xi, . . . ,x d ) 

freely generated by Xi,...,x d . Denote by rep„(^4) the category of its n-dimensional represen- 
tations. For a representation ip 6 rep n (A) we assign the point h v — (ip(xi), . . . , ip{x<i)) from 
H = (F™ xn ) d . Conversely, any point h in H determines some representation of A. Two repre- 
sentations if and ip are isomorphic if and only if h v becomes under a basis change of F™, i.e., 
GL(n) ■ h v — GL{n) ■ h^. Hence, there is one to one correspondence between classes of isomorphic 
representations and orbits of GL(n) on H . 

Proposition 3.7. A representation ip £ rep„(.4) is semisimple if and only if the orbit GL(n) ■ h v 
is closed in H . 

Suppose an algebraic group G acts on the affine variety H. 

Definition 3.8 (of a categorical quotient). A pair (Y, tty), where Y is an algebraic variety and 
■Ky is a morphism of H into Y , is called a categorical quotient for the action of G on H if for any 
morphism of algebraic varieties £ : H — > Z that is constant on the orbits of G there exists a unique 
morphism n : Y — > Z such that £ = r/ o ny • 

Denote by H//G the set of all closed orbits of G on H . 

Proposition 3.9. 

a) The set H/ /G can be endowed with the structure of affine variety in such a way that H/ /G 
becomes a categorical quotient for the action of G on H ; 

b) ¥[H//G] = ¥[H} G . 

Corollary 3.10. Let ip, tp € rep„(,4) be semisimple representations. Then p ~ ip if and only if 
f(h v ) = f(h^)forallfe¥[H] G . 

3.3 Representations of free algebras with involution 

In this section we assume that F is an algebraically closed field of characteristic different from two. 
Then results from the previous section can be proven for algebras with involution. Let us state it 
in a precise way. 

An algebra B with unity is called an algebra with involution * if 
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• * : B B is a linear map; 



• 1* = 1, where 1 stands for the unity of £>; 

• (xy)* = y*x* for all x,y € B; 

• x** — x for all x e B. 

So we assume that * is identical on F C B. 

An e-form (where e = ±1) on a vector space V — F" is a bilinear nondegenerate form (•, •) : 
V ® V — > F which is e-symmetric, i.e., (u, v) — e(v,u) for all u,v 6 V. Given an e-form on 
V, the algebra EndF(V^) can be endowed with a canonical involution defined by the property: 
(f*(u),v) = (u,f(v)) for all u, v s V and / 6 Endp(V). Since any nondegenerate symmetric 
(skew-symmetric, respectively) bilinear form is isomorphic to the bilinear form defined by the 
matrix E (J, respectively), we can assume that a canonical involution on EndF(V) is one of the 
following involutions: 

a) A^A T , 

b) A -> - JA T J, 

where A £ End^V^) ~ jr™x™_ These involutions are called orthogonal and symplectic, respectively. 

An n-dimcnsional representation of the algebra with involution B is a ip € rep ?l (£>) (i.e., 93 : 
£> — > EndF(V A ), where V = F", is a representation of B considered as a usual algebra) such that 

• Endp(V) is endowed with canonical involution which we also denote by *; 

• ip(x*) — ip(x)* for all x e B. 

So we consider two kinds of representations: orthogonal and symplectic. Denote the categories 
of these representations by rep°(£5) and rep| p (i3), respectively. Let G stands for 0(n) in the 
orthogonal case and for Sp(n) in the symplectic case. 

Consider a free associative algebra B — ¥(xi, . . . , Xd, yi, ■ ■ ■ , yd) & n d endow it with involution 
as follows: x\ = yi, . . . , x*j — yd- Then B is called a free algebra with involution. 

As in the previous section, for a ip £ rep°(£>) we assign the point h v € H = (F" x ™) d . Then we 
can see that there is one to one correspondence between classes of isomorphic representations from 
rep°(£>) and orbits of G on H. Moreover, analogues of Propositions 13. 7\ 13.91 and Corollary 13.101 
are valid. Similar results are also valid for symplectic representations. 

3.4 The definition of cr t r 

Assume that X, Y, Z are n x n matrices and t, r > 0. To describe relations between 0(n)-matrix 
invariants, we need the polynomial a t ,r{X,Y, Z). In order to define it, consider the quiver Q 




where there are two arrows from vertex 1 to vertex 2 and there are two arrows in the opposite 
direction. By abuse of notation arrows of Q are denoted by the same letters as matrices. Hence, 
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any path in Q (see Section 12. 3p can be interpreted as a product of matrices. Note that for any 
a G Qi an arrow a T € Qi is uniquely defined. 

Let a = a.\ ■ ■ ■ a p and /3 = (3\ ■ ■ ■ j3 q be closed paths in Q, where a%, . . . , a p , 0x,...,/3 q G Qi. 
Then 

• we say that a and /3 are egwa/ and write a = (3 if p = g and a% = f3i, . . . , a p = P p ; 

• a T is a closed path in Q defined by a T = a J ■ • ■ Off; 

• we say that a and /3 are equivalent and write a ~ if there exists a cyclic permutation 
7r G S p such that a 7r ( 1 ) • • ■ a Tr (p) — or a w(i) • ' ' a ?r(p) = /3 T - 

As an example, we have (IF T Z) T = Z T YX T , YZ ~ F T Z T , 171 r Z ~ iy T I r Z T . 

Denote the degree of a path a in (3 G Qi by deg /3 (a) and the multidegree of a path a by 

mdeg(a) = (deg x (a) +deg XT (a), deg y (a) +deg yT (a), deg z (a) +deg z r(a)). 

As an example, Aeg Y {YZ) = 1, deg YT (YZ) = 0, and mdeg(XYX T X T Z T ) = (3, 1, 1). 

Definition 3.11 (of at,r(X,Y, Z)). Let V be a set of representatives of equivalence classes of 
primitive closed paths in Q (see Section [2~B1 for the definition). Then 

a t , r (X,Y,Z) = £(-l)'+:£?=iMdeg y (aO+deg z (a 4 )+l) . . . ^ ( Qp ), 

where the sum ranges over pairwise different ai, . . . , a p € V and ji, ■ ■ ■ ,j p > 1 such that 

ji mdeg(ai) H 1- j p mdeg(a p ) = (t,r,r). 

For t = r = we define a ofi (X, Y, Z) = 1. 

Example 3.12. 1. If t = and r = 1, then 

p = {rz, YZ T , ...}. 

Hence a ,i(A, F, Z) = - tv{YZ) + tr(YZ T ) 
2. Ut = r = l, then 

V = {X, YZ, YZ T , XYZ, XYZ T , XY T Z, XY T Z T , . . .} 

and we can see that ai } i(X, Y, Z) = 

- tr(A) tr(FZ) + tr(X) tr(YZ T ) + tr(XYZ) - tr(XYZ T ) - tr(XY 7 + tr{XY T Z T ). 

Remark 3.13. <r t , (X,Y,Z) = a t (X). 

Remark 3.14. The complete linearization of <Jt,r was introduced by Procesi (see [31], Section 8 
of Part I). Then a t , r was introduced by Zubkov in [H] . Note that we have a' t+2ryr (X, Z, Y) = 
<r t ,r(X,Y, Z), where u' t+2rr stands for the function defined in The mentioned definitions 
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from [3T] and [H] are different from our definition, but their equivalence can be established applying 
the decomposition formula from [26j . 

Another way to define a t . r is via the determinant-pfaffian DP r r (X, Y, Z) that was defined in [30] 
as a "mixture" of the determinant of X and pfaffians of Y and Z. See below part 4 of Example l6.8l 
for the definition. One can show that DP r r relates to Ot, r in the same way as the determinant 
relates to at, i.e., for and n x n matrices X, Y, Z we have 

• BP r , r (X + XE, Y, Z) = YftLo X to ^(Jt,r(X, Y, Z), where n = t + 2r, t > 0; 

. det(X + XE) = Y, n t=0 A n "V t (X). 

In particular, if n = t + 2r, then DP rir (X, Y, Z) = at, r (X,Y, Z). Note that this approach gives 
us (it,r(X, Y, Z) as a polynomial in entries of matrices X, Y, Z. But for our purposes we have to 
present Gt^iX^Y^Z) in a different way, namely, as a polynomial in ut(a), where t ranges over 
positive integers and a ranges over monomials in X, Y, Z, X T ', Y T , Z T . 

3.5 Relations for matrix invariants 

In this section we use notations from Section |2~T1 We assume that G is GL(n) or 0(n). We denote 
by M. the monoid freely generated by letters x\, . . . , Xd (x\, . . . , Xd, xf , . . . , xj, respectively) if 
G = GL(n) (G = 0(n), respectively). Given a = otx ■ ■ ■ a s G M, where a.% — Xi or cti = xf , we 
assume X a = X ai ■ ■ ■ X as , where 




Xi , if a, t = x t 



Let us recall that a G M. is primitive if a is not equal to a power of a shorter monomial. 
Lemma 3.15. 

a) The algebra ¥[H] is generated by at(X a ), where 1 < t < n and a G M. is primitive. 

b) For ai G F, a, G M., and t > the element otQ^. aiX ai ) is a polynomial in ai(X a ), where I 
ranges over {1, . . . , n} and a ranges over primitive elements from M. 

c) Let G = 0(n), ai,bj,Ck G F, at,Pj,jk G M, and t,r > 0. Then the ele- 
ment o~t,r(Vli (LiXai, ^jbjXp^ X)fe c fc^7fc) * s a polynomial in o~i(X a ), where I ranges over 
{1, . . . , n} and a ranges over primitive elements from Ai. 

Proof. In this proof A, A\, . . . , A p stands for an arbitrary n x n matrices, 
a) For 1 < t < n and I > 2 we have the following well-known formula: 

a t (A l )= b^ Hi a l {Ar---a tl {Ar, (4) 

U,...,iti>0 

where we assume that cri(A) = for % > n. In (0| coefficients bf^ ^ G Z do not depend on A and 
n. If we take A = diag(ai, . . . , a n ) is a diagonal matrix, then a t (A l ) is a symmetric polynomial 
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in a%, . . . ,a n and o~i(A) for 1 < i < n is the i th elementary symmetric polynomial in a±, . . . , a n . 
Thus, the coefficients i i with rl < n can easily be found. As an example, 

tr(A 2 )=tr(A) 2 -2a 2 (A). (5) 

Using (|4]) we complete the proof. 

b) We start with a definition. Let yx, . . . ,y p be some letters. Then a cycle in these letters is an 
equivalent class of some monomial in yi y p with respect to cyclic permutations. 

For 1 < t < n Amitsur's formula states [T]: 

a t (Ai +--- + A p ) = J^i-iy-^-^vM ■ ■ ■ <T jg (c q ), (6) 

where the sum ranges over all pairwise different primitive cycles c\, . . . , c q in letters Ai, . . . , A p 
and positive integers ji, . . ■ ,j q with J2i=iJi deg(ci) = t. As an example, 

a 2 (A 1 + A 2 ) = a 2 (A 1 ) + a 2 {A 2 ) + tr(Ai) tr(A 2 ) - tr(AiA 2 ). 

Using ([?]) together with the formula 

<r t {aA) = oV t (i4), (7) 

where a G F, we complete the proof. 

c) Follows from part b) . □ 

For ai, bj, Ck G F, a, /?, a^, /3j,jk G and t, r > we consider the following equalities: 

a) a t (X a X fj ) - a t (XfjX a ) = for 1 < t < n; 

b ) ^t(Ej = for t > n; 

c ) o't.rCZ), ai^ai) J2j bjX^, J2k °kX lk ) = for t + 2r > n. 

d) free relations, i.e., relations between 0(m)-invariants that are valid for any m > 0. 

Here we assume that Lemma[XT5]is applied to these equalities, i.e., the left hand sides of the above 
equalities are polynomials in ai(X a ), where / ranges over {1, . . . ,n} and a ranges over primitive 
elements from Ai. 

Theorem 3.16. Consider the system of generators for ¥[H] G from part a) of Lemma \3.15[ Then 
the ideal of relations between them is generated by the following relations: 

• a) and b), if G — GL(n); 

• a), c), and d), if G = 0{n). 

Remark 3.17. Remark 13.131 implies that relation c) with r = is the same as relation b). 

Remark 3.18. In case of characteristic zero relations for matrix GL(n)-, 0(n)- and Sp(n)- 
invariants were described by Procesi in |31j . Independently, relations for G — GL(n) were es- 
tablished by Razmyslov in [33]. Over a field of arbitrary characteristic relations for matrix GL{n)- 
invariants were described by Zubkov (see [39]) and relations for matrix 0(rt)-invariants were de- 
scribed by Lopatin (see [IS])- The last result is based on the reduction to mixed representations 
of quivers that was performed by Zubkov in [44] . 
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4 Representations of quivers 
4.1 Definitions 

The notion of a representation of a quiver generalizes the notion of several linear maps on a vector 
space (see Section I3~T|) . 

Consider a quiver Q = (Qo, Qi) and assume that Qo = {1, . . . , 1} for some Given a dimension 
vector n = (n±, . . . ,n{), we assign an n^-dimensional vector space V v to v £ Qo- We identify V v 
with the space of column vectors ¥ nv . Fix the standard basis e(v,l), . . . , e(v,n v ) for F"", where 
e(v, i) is a column vector whose i th entry is 1 and the rest of entries are zero. A representation of 
Q of dimension vector n is a collection of matrices 

h = (h a ) aeQl EH = H(Q,n) = F"«' x "«" ~ Hom F (F a », V a >), 

ueQi a£Qi 

where F™ lX ™ 2 stands for the linear space of n\ x ni matrices over F and the isomorphism is given 
by the choice of bases. The action of the group 

G = GL(n) = Yl GL{n v ) 

veQo 

on H is via the change of the bases for V v (v 6 Qo). In other words, GL(n v ) acts on V v by left 
multiplication, and this action induces the action of G on H by 

g ■ h = (g a 'h a g^) aeQll 

where g = {g a )aeQi £ G and h = (h a ) a eQi £ H. We refer to the pair (Q, n) as the quiver setting. 

The coordinate ring of H (i.e. the ring of polynomial functions / : H — > F) is the polynomial 
ring 

¥[H] = ¥[ Xij (a) \a G Qi, 1 < * < n a /,l < j < n a //]. 

Here ^ (a) stands for the coordinate function on that takes h € H to the (i, j) th entry of the 
matrix h a . Denote by 

\ \ 

%n n r ,1 

(a) ••■ (a) 

the n a r x n Q /< generic matrix. 

The action of G on H induces the action on ¥[H] as follows: (g ■ f)(h) = J(.g _1 • h) for all 
g e G, / £ F[H], he H. In other words, 

5 ' = (*) J)' 1 " entl T °f 9a^ X a9a" 

for all 5 £ G, a G Qi. The algebra of invariants is 

¥[Hf = {f G F[fl] | 5 • / = / for all 5 G G}. 

Example 4.1. Let Q be a quiver 

a 

©^ ^® 
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and n — (n, m). Then the group G = GL(n) x GL(m) acts on H = F" xm by the rule 

(91,92) ■ A = giAg^ 1 , 

where (31,32) G G and A £ F nxm . Hence the orbits of this action correspond to linear maps from 
F m toF".' 

Example 4.2. Let Q be a quiver with one vertex and d loops: 




and n = (n). Then the group G = GL(n) acts on H — (F nx ™) d by the diagonal conjugation 
(see ([3])). Obviously, the orbits of this action correspond to d-tuples of linear maps on F™. Gener- 
ators for F[ii] G were described in part a) of Theorem 13. II 

Example 4.3. Let Q be a quiver 

a 

and n = (n, m, I). Then the group G = GL(n) x GL{m) x GL(l) acts on H = F" xm F mx/ by 
the rule 

(31,32,53) • (-4,-8) = (g 1 Ag^ 1 ,g 2 Bg^ 1 ). 

4.2 Generators for invariants of representations of quivers 

Let us recall that the notion of closed path was given in Section 12.31 

Theorem 4.4. Let (Q, n) be a quiver setting. Then ¥-algebra ¥[H(Q, n)] GL ^ is generated by 

at{Xfr ■ ■ ■ Xp s ), 
where /3\ ■ ■ ■ f3 s is a closed path in Q and 1 < t < . 

This theorem implies that if there is no closed paths in Q, then 

W[H(Q,n)] GL( - n ) =¥. 

Remark 4.5. The system of generators from Theorem 14.41 is infinite but it can be shown that 
there exists a finite generating system. As an example, invariants of quivers of dimension (2, . . . , 2) 
were considered in [27], where an upper bound on degrees of elements of a minimal system of 
generators was given and its precision was estimated. 

Remark 4.6. Over a field of characteristic zero, Theorem 14 . 41 was proven by Le Bruyn and Procesi 
in [25]; over a field of arbitrary characteristic, it was proven by Donkin [IB] . 
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4.3 Relations for invariants of representations of quivers 

Relations for invariants of quivers can be described in a similar way as relations for matrix GL(n)- 
invariants (see Section [3. 5p . In what follows we will need the notions of primitive path and incident 
path that were introduced in Section [2.31 

We assume that (Q,n) is a quiver setting. Let Ai be the set of closed paths in Q. For 
P = pi" ■ /3 S £ Ai, denote Xp = Xp 1 ■ ■ ■ X@ B . The following lemma is an analogue of Lemma 151 

Lemma 4.7. 

a) The algebra ¥[H (Q, n)] GL ^ is generated by ot(Xp), where P = Pi ■ ■ ■ p s € M is primitive 
and 1 < t < npi^ . 

b) Let a,i £ F, cii £ Ai be incident to v £ Qq, and t > 0. Then the element o' t (^2, i aiX ai ) is a 
polynomial in ai{X a ), where I ranges over {1, . . . , n v } and a ranges over primitive elements 
from Ai. 

Given a vertex v € Qq, we consider the following equalities: 

a) a t (X a Xp) — at{X/jX a ) = for 1 < t < n v , where a,P are paths in Q such that a/3 € M. is 
incident to v; 

b) o~ti^2, i aiX ai ) = for t > n„, where a t £ ¥ and on £ M. is incident to v. 

Here we assume that Lemma |4~71 is applied to these equalities, i.e., the left hand sides of the above 
equalities are polynomials in ai(X a ), where I ranges over {1, . . . ,n v } and a ranges over primitive 
elements from M.. 

Theorem 4.8. Consider the system of generators for ¥[H(Q,n)] GL ^ from part a) of Lemma \4-7\ 
Then the ideal of relations between them is generated by relations a) and b) considered for all 
vertices of Q. 

Remark 4.9. Theorem 14.81 was proved by Zubkov in [4"2"] . 

5 £}-mixed representations of quiver 
5.1 Definitions 

The notion of representations of quivers can be generalized by the successive realization of the 
following steps. At the end of this procedure we obtain mixed representations of quivers. 

1. Instead of GL(n) we can take a product G(n,g) of classical linear groups. Here g = 
(g 11 . . . ,gi) is a vector, whose entries g 1 , . . . , are symbols from the list GL, O, Sp, SL, SO. 
By definition, 

G{n,g)= J] Gv, 

v&Qo 
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where 



' GL(n v ), ifg 
0{n v ), if flr 



V 



V 



GL 

O 

Sp 

SL 

SO 



G v = { Sp(n v ), if g 



V 



SL(n v ), ifg 
. SO(n v ), ifg 



V 



V 



Obviously, we have to assume that n and g are subject to the following restrictions: 

a) if g v = Sp (v 6 Q ), then n v is even; 

b) if g v is O or SO (v e Q ), then the characteristic of F is not 2. 

2. We can change the definition of G(n, g) in such a manner that allows us to deal with bilinear 
forms together with linear mappings. Since bilinear forms on some vector space V are in one 
to one correspondence with linear mappings from the dual vector space V* to V, we should 
change vector spaces assigned to some vertices to the dual ones. In order to do this consider 
a mapping i : Q —* Qo such that 

c) i is an involution, i.e., i 2 is the identical mapping; 

d) = n v for every vertex j; G Qo- 

For every v £ Qo with v < i(v) assume that V^ v ) = V*. Consider the dual basis 
e(v, 1)*, . . . , e(v,n v )* for V* and identify V* with the space of column vectors of length 
n v , so e(v, i)* is the same column vector as e(v, i). 

The action of GL(n v ) on V v induces the action on V*, which we consider as the degree one 
homogeneous component of the graded algebra ¥[V V ]. Given g v € GL(n v ) and u £ y„*, we 
have 



Hence, we should change the group G to 

G(n, g, i) = {g G G(n, g) \ g l{v) = (g^ 1 ) 7 for all v £ Q with v < i(v)}. 

Since the vector spaces F™ and (F™)* arc isomorphic as modules over 0(n), Sp(n), and SO(n), 
we assume that 

e) if g v is O, Sp or SO (v e Q ), then i(v) = v. 

3. Instead of the space H{Q 1 n) we should take its subspace H(Q,n,h), where h = (h a ) ae Q 1 
and h a is a symbol from the list M, S + , S~ , L + , L~ . By definition, 



9vu= (g v r ) T u. 



H(Q,n,h)= H t 



a > 



t>6Ql 



where 



' F n a ,xn a „ 5 if ha = M 

S+(n a >), if h a = S+ 



H a = < S (n a /), if h a = S . 
L + (n Q '), if h a = L + 
L-(n a/ ), if h a = L~ 



Additionally, we have to assume that n and h are subject to the restriction: 
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f) if h a ^ M {a € Qi), then n a i — n a /i. 

Consider a group G — G(n, g, 1) C GL(n) and a vector space H — H(Q, n, h) C H(Q, n) satisfying 
the previous conditions a)-f). To ensure that these inclusions induce the action of G on H, we 
assume that the following additional conditions are also valid for all v 6 Qq, a £ Q\\ 

g) if a is a loop, i.e., a' — a", and h a is S + or S~ , then g a , is O or SO; 

h) if a is a loop and h a is L + or L~ , then gr Q / = Sp; 

i) if a is not a loop and h a ^ M, then i(a') = a" and /i Q is S* + or S~ . 

A quintuple O = (Q,n,g, h,i) satisfying a)-i) is called a mixed quiver setting and elements of H 
are called Q-mixed representations of the quiver Q. Definitions of the generic matrices X a and 
the algebra of invariants ¥[H] G are the same as above. Note that if h a = S + , then = X a ; 
if h a = S~, then X^ — —X a ; if h a — L + , then (X a J) T = X a J; and if h a = L~ , then 
(X a J) T = —X a J. 

Example 5.1. 1. Let Q be the following quiver 

a 

Define a mixed quiver setting Q = (Q, n. g. h, i) by ni = i%2 = n, g x = g 2 = GL, h a = hp = S + , 
and i(l) = 2. The group G(n,g,i) ~ GL(n) acts on H(Q,n,h) = S + (n) © S + (n) by the rule 

g-(A,B) = (gAg T ,gBg T ) 

for g G GL(n) and (A, £?) G S + (n) © S + (n). Hence the orbits of this action correspond to pairs of 
symmetric bilinear forms on F™. If we put h a = hp = S~ , then we obtain pairs of skew-symmetric 
bilinear forms on F n . The classification problem for such pairs is a classical topic going back to 
Weierstrass and Kronecker (see [H[ and [10]). 
2. Let Q be the following quiver 




Define Q = (Q,n,g, h, i) by ri\ = n 2 = n, g 1 = g 2 = GL, h a — hp = h 1 = AI, and i(l) = 2. The 
group G(n,g,i) ~ GL(n) acts on H(Q,n,h) — (F" x ™) 3 by the rule 

g-(A,B,C) = (gAg T , 9 B 9 T ,gCg- 1 ). 

Hence the orbits of this action correspond to pairs of bilinear forms on F" together with a linear 
map on F n . 

3. Let Q be the following quiver 
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Define a mixed quiver setting = (Q, n, g, ft,, 1) by ni = ri2 = n, ns = to; g 1 = g 2 = GL, g 3 = O; 
h a = h.y = M, hp = S + ; and i(l) = 2, i(3) = 3. Hence the action of G(n,g,i) ~ GL(n) x 0(m) 
on H(Q, n, ft) = F nxm © S+(n) © F mx ™ is given by 

( 9l ,g 3 ) ■ (A,B,C) = (giAgJ,(g^) T Bgi\g3Cgl) 

for (51,33) £ GL(n) x O(m) and (A,B,G) e F" xm © S*+(n) © F mxn . 

Let us consider some partial cases of the notion of 0-mixed representations. 

Example 5.2. We assume that = (Q,n,g,h,\) is a mixed quiver setting. 

• If g v = GL and i(v) = v for all v e Q , then /i Q = M for all a € Qi; hence in this case 
O-mixed representations are the usual representations of Q. 

• If € {Gi, O} for all w £ Qo and h Q e {M, S 1- } for all a 6 Qi, then 0-mixed representa- 
tions are orthogonal representations from [5] . 

• If g v E {GL, Sp} for all v e Q and h a € {M, S + } for all a € Qi, then 0-mixed representa- 
tions are symplectic representations from [5]. 

• If g v = GL for all v E Qo an d h a = M for all a £ Qi, then 0-mixed representations are 
mixed representations from [40j , [43j . 

• If g v E {GL, O, ^1} for all v E Qo, then 0-mixed representations are supermixed representa- 
tions of quivers from [40] , [43] , or equivalently, representations of signed quivers from [34] . 

Example 5.3. Let = (Q,n,g,h,i) be a mixed quiver setting. We put G = G(n,g,i) and 
H = H(Q,n,h). 

• If g v = SL and i(v) — v for all v E Qo, then h a = M for all a £ Q\. In this case F[_ff] G is 
the algebra of semi-invariants of (usual) representations of Q. 

• If g v = SL for all v E Qo and h a — M for all a E Qi, then F[i7] G is the algebra of 
semi-invariants of mixed representations of Q. 

• If g t , £ {SL,SO} for all t> E Q and h, Q E {M, S + , S~} for all a E Qi, then F[iJ] G is the 
algebra of semi-invariants of supermixed representations of Q. 

5.2 Generators for invariants of supermixed representations of quivers 

At first we consider an arbitrary mixed quiver setting and present some definitions. Then we 
restrict the general case to supermixed representations of quivers (see Theorem 15. 6p . The general 
case is considered in Section 17.21 

Let = (Q,n,g, h, 1) be a mixed quiver setting and Qo = {1, . . . .1}. Without loss of generality 
we can assume that 

if v £ Qo and g v — GL or SL, then i(v) ^ v. (8) 
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Otherwise we can add a new vertex v to Q, and set i(u) = v, n- — n v , g w = g v ; this construction 
changes neither the space H(Q, n, h) nor the algebra of invariants. 

Definition 5.4 (of the mixed double quiver setting Q D ). Define the mixed double quiver setting 
Q D = (Q D ,n,g,h D ,i) as follows: 

Qo = Qo, Q? = Qi ]]_{a T | a 6 Qi, h a — M}, 

where (a T )' — i(a"), (a T )" = i(a'), and h^r — M for a G Qi with h a = M and — h a for all 
a G Qi. 

Define a mapping $ D : F[i/(Q Z5 , n, -> F[ff(Q, n, h)] such that 

a) for a G Qi we have $ c (I a ) = X Q ; 

b) for a € Qi and h a — M we define $ D (X Q r) as follows: 

. If g a , ^ and g a „ ? Sp, then $ d (X q t) = X%. 

. H 5a , - Sp and 9ct „ ^ then <S> D (X aT ) = Xjj(n Q /). 

. If flQ , ^ and g a „ = Sp, then $ d (X q t) - J(n a „)X^. 

• If g a , = Sp and gr Q „ = Sp, then $ d (X q t) = J(n a „)X^ J(n a >). 

Here <I> D (X Q ) stands for the matrix, whose (i,j) th entry is $> D (xij(a)). 
Example 5.5. Let Q be 

a 

®^^© © 

Define a mixed quiver setting Q = (Q,n,g,h,i) by i(l) = 2, i(3) = 4, i(5) — 5; g ± = g 2 = GL, 
9.3 = 94 = SX, 9 5 = O; h a = h 7 = h 5 = M, hp = S+. Then Q D is 



T 




The following theorem was proven by Zubkov in [43] . 

Theorem 5.6. Let (Q,n,g,h,i) be a mixed quiver setting satisfying |f5|). If g v G {GL,0,^3} /or 
w G Qo, then W-algebra ¥[H(Q,n,h)] G{ ~ n ' 9 ^ is generated by 

<S> D (a t (X Pl ---X Ps )), 

where fti ■ ■ ■ f3 s * s a closed path in Q D and 1 < t < rip . 

Remark 5.7. For any system of generators from Theorem 15.61 the exists a finite subset that 
generates the corresponding algebra of invariants. 

Note that Theorem 15.61 implies all parts of Theorem 13.11 but part d) . 
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5.3 Relations for invariants of mixed representations of quivers 

In this section we assume that = (<2, n, g, h, 1) is a mixed quiver setting with 

g v = GL for all v € Qq and h a = M for all a G Q\. 

Then l}-mixed representations are called mixed representations (see Example I5.2[) . For short, we 
denote H — H(Q, n, h) and G — GL(n, g, 1). 

Denote by M the set of closed paths in the double quiver Q D (see Definition 15. 4|) . Assume 
{ex., (3, 7) is a triple of paths in Q D ; then it is called incident to v G Qq if v < i(v), a is incident to 
v, (3 is a path from i(v) to v, and 7 is a path from v to i(w). In what follows we will also need the 
notion of primitive path that was introduced in Section [2.31 

The following lemma is an analogue of Lemma 13.151 

Lemma 5.8. 

a) The algebra ¥[H] G is generated by o~ t (X a ), where 1 < t < and a E M is primitive. 

b) If at G F, oti G M. is incident to v G Qq, and t > 0, then the element c r t(^2 i aiX ai ) is a 
polynomial in ai(X a ), where I ranges over {1, . . . , n v } and a ranges over primitive elements 
from M . 

c) Let ai,bj,Ck G F, (a$, /3j, 7^) be a triple incident to v G Qq, and t,r > 0. Then the ele- 
ment cr t . r ('^2 i aiX ai , ^jbjXpj, Xlfc ^-^Yfc) * s a polynomial in ai(X a ), where I ranges over 
{1, . . . , n v } and a ranges over primitive elements from M.. 

Given v G Qo, we consider the following equalities: 

a) at{X a Xp) — o-t(XpX a ) = for 1 < t < n v , where a, (3 are paths in Q D such that a/3 G Ai 
is incident to v; 

b) o~t(^2 i a-iX ai ) = for t > n v , where Ui G M. is incident to v; 

c ) ^rEi a i-^Qii) J2j bjX^, J2k °kX lk ) = for t + 2r > n, where u < i(v) and (a i)( 3j-,7 fc ) is a 
triple incident to 1; for all k. 

d) free relations, i.e., relations between generators of F[_ff] G that are valid for any dimension 
vector n. 

Here we assume that Lemma IS~8l is applied to these equalities, i.e., the left hand sides of the above 
equalities are polynomials in ai(A), where I ranges over {1,. .. ,n v } and a ranges over primitive 
elements from A4. 

Theorem 5.9. Consider the system of generators for ¥[H] from part a) of Lemma \5.8l Then 
the ideal of relations between them is generated by relations a), b), c), and d) considered for all 
vertices of Q. 

Remark 5.10. Theorem 15.91 was proved by Zubkov in [33] . 



20 



6 Tableaux with substitutions and bpf 



Definition 6.1 (of a block partial linearization of the pfaffian). Fix n = {n\, . . . , n m ) G N m , where 
n = ni + • • • + n rn is even. For any 1 < p, q < m and an n p x n q matrix X denote by X p,q the 
n x n matrix, partitioned into m x m number of blocks, where the block in the (i, j) th position is 
an ni x nj matrix; the block in the (p, q) th position is equal to X , and the rest of blocks are zero 
matrices. 

Let 1 < pi, . . . ,p s , q%, . . . ,q s < m, let Xj be an n Pj x n qj matrix for any 1 < j < s, and let 
r%, . . . , r s be positive integers, satisfying r% + • • • + r s = n/2. The element 

^ , -v[ " : vr ) (9) 

is a partial linearization of the pfaffian of block matrices Xf 1 ' 91 , . . . , Xf s ' 9s , and it is called a block 
partial linearization of the pfaffian (b.p.l.p.). 

We consider only b.p.l.p.-s, satisfying 

Tj + rj = ni for all 1 < i < m, (10) 

1<j<s, Pj=i l^J^ s i Qj— * 

since only these elements appear in the context of the invariant theory. 



Example 6.2. Let n = (5, 3) and let X\, X2, X3 be 5 x 3, 5 x 5, and 3x3 matrices respectively. 
Then 

Y l,2 _ ( X 1 \ 1,1 _( X 2 \ 2 , 2 _ / \ 

Al ' - ^ J' ^ 2 - ^ oj' 3 "^0 x 3 j 

are 8x8 matrices. The b.p.l.p. / = pf 1 2 1 (Xj L ' 2 , X 2 ' , X^ 2 ) satisfies condition (fT0| . It is convenient 
to display the data that determine / as a two-column tableau filled with arrows: 



a - 




I 






1 




i- 


c 






1 





The arrow a determines the block matrix X\ y2 as follows: a goes from the 1 st column to the 2 nd 
column and we assign the matrix X\ to a. To determine the block matrix X^ 1 we take two arrows 
b, c that go from the 1 st column to the 1 st column, since the degree of / in entries of X2 is 2. Assign 
the matrix X2 to b and to c. Finally, the arrow d determines the block matrix X 2 ' 2 - d goes from 
the 2 nd column to the 2 nd column and X2 is assigned to it. Condition (110|) implies that for every 
column the total amount of arrows that start or terminate in the column is equal to the length of 
the column. Note that this conditions do not uniquely determine arrows of a tableau. 
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Below we formulate the definition of a tableau with substitution that gives alternative way to 
work with b.p.l.p.-s. Acting in the same way as in Example 16.21 for every b.p.l.p. / given by ([9]) 
we construct a tableau with substitution (T, (X±, . . . , X s )) and define the function bpf r such that 
f = ±b V f T (X 1 ,...,X s ). 

Definition 6.3 (of shapes). The shape of dimension n — (ri\, . . . , n m ) 6 N m is the collection of m 
columns of cells. The columns are numbered by 1,2, ... , to, and the z th column contains exactly 
rii cells, where 1 < i < to. Numbers 1, . . . , ni are assigned to the cells of the i th column, starting 
from the top. As an example, the shape of dimension n = (3, 2, 3, 1, 1) is 



1 2 3 4 5 



1 


1 


1 


1 


1 


2 


2 


2 






3 




3 







Definition 6.4 (of a tableau with substitution) . Let n = (ni, . . . , n m ) € N m and let n = ni + • • • + 
n m be even. A pair (T, (X\, . . . , X s )) is called a tableau with substitution of dimension n if 

• T is the shape of dimension n together with a set of arrows. An arrow goes from one cell of 
the shape into another one, and each cell of the shape is either the head or the tail of one 
and only one arrow. We refer to T as tableau of dimension n, and we write a £ T for an 
arrow a from T . Given an arrow a S T, denote by a' and a" the columns containing the 
head and the tail of a, respectively. Similarly, denote by 'a the number assigned to the cell 
containing the head of a, and denote by "a the number assigned to the cell containing the 
tail of a. Schematically this is depicted as 

a" a 1 







7 

a 





• (p is a fixed mapping from the set of arrows of T onto [l,s] that satisfies the following 
property: 

if a, b € T and tp(a) ~ (p(b), then a' = b', a" = b"\ 

• (X\, . . . ,X S ) is a sequence of matrices such that the matrix X^ a ^ assigned to the arrow 
a G T is n a " x n a / matrix and its (p, q) th entry is denoted by (Xj) pq . 

Example 6.5. Let T be the tableau 



a 


b 




-+- 


— ) 

c 




d- 
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of dimension (3,3,2). Define <p by <p(a) = 1, tp(b) = tp(c) = 2, and <p(d) = 3, and let X\, X 3 be 
3x3 matrices and X2 be a 3 x 2 matrix. Then (T, (Xi, X2, X3)) is a tableau with substitution. 

Definition 6.6 (of bpf T (Xi, .... X s ))). Let (T, (Xi, . . . , X s )) be a tableau with substitution of 
dimension n. Define the polynomial 

bpf r (Xi, . . . ,X S ) = ^2 Sgn(TTi) • • -Sgn(7T m ) (^,( a ) )*•,,,/ ("»W('o)' (H) 

7ri£5 ni ,...,TT m eS nm aeT 

and the coefficient 

s 

CT = Y[Ma^T\^a)=j}l 

3=1 

In the case F = Q define 

bpf T (Xi, . . . , X s ) = — bpf T (X u X s ). 
cr 

Since bpf T (Xi, . . . , X s ) is a polynomial in entries of X±, . . . , X s with integer coefficients, the defi- 
nition of bpf 7 -(Xi, . . . , X s ) extends over an arbitrary field. 

The next lemma shows that bpf is a b.p.l.p. 

Lemma 6.7. 

a) Let f be a b.p.l.p. 0) satisfying \10\) . Then there is a tableau with substitution 
(T, (Xi, . . .,X S )) such that bpf T (Xi, ...,X S ) = ±f. 

b) For every tableau with substitution (T, (Xi, . . . , X s )) of dimension n there is a b.p.l.p. f 
satisfying L?fl|) such that bpfq-(Xi, .... X s ) = ±f. 

Proof, a) A tableau with substitution (T, (Xi, . . . , X s )) of dimension n is constructed as follows. 
Its arrows are Oj>, where ey r goes from the p^ to the g* h column for I < j < s and 1 < r < kj. 
Condition (jlOp guarantees that for any 1 < i < m the total number of arrows that begin or end in 
the i th column is n^. Complete the construction by setting <p(aj r ) = j. Note that (T, (Xx, . . . , X s )) 
is not uniquely determined by /. 

Since Xf 1,qi , . . . , Xf s ' qs are block matrices, the permutation ir from formula ([2]) can be written 
as the composition n = %x o ■ ■ ■ o 7r m , where the permutation 7^ acts as an identity on the set 
[I, Ci-i] U [cj+i + f , n/2] ioi Ci — ki + ■ ■ ■ + ki (1 < i < m). The claim follows from formula 

b) Consider 01, . . . , a s € T such that tp(ai) = I, . . . , <p(a s ) = s. Then 

b P f r (x 1; ...,x s ) = ±pl rit ^ rs (xf' a '\. . .,X^'<), 

where rj — #{a € T | ip(a) = j} for any 1 < j < s. □ 
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Example 6.8. 1. Let n be even and (T, (Xi , X s )) be a tableau with substitution of dimension 
(n), where T is 

-h 



In other words, arrows of T are ai, . . . , a, i y 2 , where 'a^ = 2i, "a; = 2i — 1, a' ; = a" = 1 for 
1 < i < 7i/2, and Xi, . . . , X s are n x n matrices. 

If s = 1, then (fTJ) implies bpf r (Xi) = pf(Xi) for F = Q and consequently for an arbitrary F. 

If s > 1, then by © 

bpf r (X 1 ,...,X s )= P f ri) ... ira (X 1 ,...,X s ), 

where Tj = #{a € T \ f(a) = j} for any 1 < j < s, is a partial linearization of the pfaffian. 

2. For n x n matrices Xi, . . . , X s let (T, (Xi, . . . , X s )) be the tableau with substitution of 
dimension (n,n), where T is 



ai - 








On" 





If s = 1, then the formula 

1 n 
det(X) = — sgn^Osgn^)]^^!^),^^): ( 12 ) 

which is valid over Q, implies the equality bpf r (Xi) = det(Xi) over every F. For s > 1 the 
expression bpf T (Xi, . . . , X s ) is a partial linearization of the determinant. 

3. If (T, (Xi,X2)) is a tableau with substitution from part 2 of Example 16. 8[ where for 
1 < k < n we have </?(oi) = • • • = ip(ak) = 1, ^{o-k+i) = ■ ■ ■ = tp(a n ) = 2, Xi = X, and X2 = E is 
the identity n x n matrix, then bpf 7 -(X, E 1 ) = Cfc(X). 

4. Suppose that t, r, s £ N, and X, y, Z, respectively, are matrices of dimensions (t + 2r) x 
(t + 2s), (t + 2r) x (t + 2r), (t + 2s) x (t + 2s), respectively. Let T be a tableau of dimension 
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(t + 2r, t + 2s) that in the case r = s is depicted as 











a t - 




bi 


Cl 


i- 


"t- 






b r 




i- 


-t- 



otherwise define T analogously. Let (T, (X, y 2Q) be a tableau with substitution, where f(di) = 1, 
y>(fy) = 2, and p(c fc ) = 3 for 1 < i < t, 1 < j < r, and 1 < k < s. Then bpf r (X,y Z) = 
qP)P r . s (X, y, Z), where q = ±1 and DP rs was introduced in Section 3 of [30]. Moreover, if r = s, 
then q = 1. 

The above mentioned polynomial DP r ,s is called determinant-pfaffian. This function inherits 
some properties from the determinant as well as from the pfaffian: 

a) DP , (X,y,Z) =det(X); 

b) DP r , s (x, y z) = pf (y) p7(z) if * = o ; 

c') T)P r>s {gX, gYg T , Z) = det(g) DP ns (X, Y, Z), where g is a (f + 2r) x (t + 2r) matrix; 
c") DP r>s (X#, y .g T Z#) = det(.g) DP r , s (X, y Z), where g is a (t + 2s) x (t + 2s) matrix. 

The following formula is the main result of [26j and it plays a crucial role in proofs of results 
on bpf and invariants. 

Theorem 6.9. (Decomposition formula: short version). 

Let (T, (X\, . . . ,X S )) be a tableau with substitution of dimension n G W n . Let 1 < q% < q% < m, 
d the vector d S N m 2 be obtained from n by eliminating the q{ h and the q^ 1 
coordinates. Then bpf T (Xi, . . . , X s ) is a polynomial in bpf-p^i, . . . , Yi) and crt(h), where 

• (D, (y, . . . , y)) ranges over tableaux with substitutions of dimension d such that Y\ , . . . , Yi 
are products of matrices Xi, . . . , X s , X± , . . . , X T ; 

• h ranges over products of matrices X%, . . . , X s , X± , . . . , Xj; 

• t ranges over [l,n qi ]. 

Moreover, coefficients of this polynomial belong to the image of TL in ¥ under the natural homo- 
morphism. 

The explicit formulation of the decomposition formula can be found in [26j . 



25 



7 Generators for invariants of £}-mixed representations of 
quiver 



In this section we describe generators for invariants of arbitrary mixed quiver setting 0. We start 
with a particular case of semi-invariants for bipartite quivers. 

7.1 Generators for semi-invariants of representations of bipartite quivers 

Given a quiver setting (Q,n), denote 

SL{n) = Yl SL{n v ). 

v€Qo 

Then ¥[H(Q, n)] si (™) is the algebra of semi-invariants of Q. 

A quiver Q is called bipartite, if every vertex is either a source (i.e. there is no arrow ending at 
this vertex) or a sink (i.e. there is no arrow starting at this vertex). 

Definition 7.1 (of a(Q, n) -tableau with substitution) . A tableau with substitution (T, (Yi, . . . , Y s )) 
of dimension n S N m is called a (Q,n) -tableau with substitution, if for some weight w = 
(wi ,...,wi) e N 1 we have 

• T is a union of m rectangular blocks B±, . . . , Bi, where Bi consists of Wi columns of length 
iH (1 < * < Os this condition is equivalent to n = (ni, . . . , rii, . . . , n; . . . , ni); 

• if a G T, then there exists an a 6 Qi such that 

a ) X a = V^j(a)) 

c) B a i contains column a" of T, 

b) B a ii contains column a' of T. 



Theorem 7.2. Let (Q,n) be a quiver setting such that Q is bipartite. Then the algebra 
of semi-invariants ¥[H(Q, n)] Si ( Tl ' is spanned over ¥ by the elements bpf D (Zi,...,Zh), where 
(D, (Zi, . . . , Zh)) is a (Q,n) -tableau with substitution. 

Let us remark that under the given restrictions on the quiver the generating set from Theo- 
rem [721 is smaller than that from Theorem 17.71 (see below). 

Remark 7.3. Generators for semi-invariants of an arbitrary quiver were established by Domokos 
and Zubkov in [7] using the methods from [12] , [13] , [39] , [42] , and, independently, by Derksen and 
Weyman in [4], [3] utilizing the methods of the representation theory of quivers. Simultaneously, 
similar result in the case of characteristic zero was obtained by Schofield and van den Bergh in 36J . 

Example 7.4. Consider the following quiver Q: 
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Let (T, (Yi, . . . , Y s )) be a (Q, n)-tableau with substitution of a weight u; = (ioi, w^, W3). Then T 
consists of three blocks B±, B2, -B3, where Bi is the rii x Wi rectangle (1 < i < 3). Schematically 
T is depicted as 



Si B2 -B3 



t^l W2 W3 

Here arrows of T are denoted by the same letters as the corresponding arrows of Q. 

Now let us to consider a concrete example. We assume that n = (1, 2, 1) and define a tableau 
with substitution (T, (Yi, . . . , Y4)) of dimension n = (1, 2, 2, 1, 1, 1) by T: 



a - 






-6 


„ c 













and by equalities Y v(q) = X a , Y v ( 6 ) = Y v(c) = Y"^ = X^. Then (T, (Yi, . . . , Y 4 )) is (Q, n)-tableau 
with substitution of the weight w — (lj 2, 3). 

7.2 General case 

Let = (Q, n, g, h, 1) be a mixed quiver setting and Qo = {1, . . . , I}. As in Section HT21 without 
loss of generality we can assume that condition |(5J) is valid. In this section we will use the notion 
of mixed double quiver setting Q D (see Definition 1 5. 4[) . 

Definition 7.5 (of a path Q-tableau with substitution) . A tableau with substitution (T, (Yi, . . . , Y s )) 
of dimension n 6 N m is called a pai/i Q-tableau with substitution, if for some weight w = 
(wi , . . . , wi) € we have 

• T is a union of to rectangular blocks B±, . . . , Bi, where Bi consists of lOj columns of length 
?ti (1 < * < Oi this condition is equivalent to n = (rii, . . . , ni, . . . , ra; . . . , w;); 

• if a S T, then there exists a path a = a\ ■ ■ ■ a r in Q (where ai, . . . , a r £ Qi) such that 

a ) A Ql • • • A Qi . = Yp(„), 

b) B Q ' contains column a" of T, 

c) -B^q,") contains column a' of T. 

Example 7.6. Consider the mixed quiver setting defined in Example 15.51 Let (T, (Yi, . . . , Y s )) 
be a path ii^-tableau with substitution of a weight w — (wi, ■ ■ ■ ,105). Then T consists of five 



27 



rectangular blocks Bi, . . . , B$, where rti x Wi block Bi corresponds to vertex i (1 < i < 5). 
Schematically T is depicted as 




Here we depicted arrows of T that correspond to paths in Q D of length one, i.e., arrows of Q D ; 
arrows of T are denoted by the same letters as the corresponding arrows of Q D . 

Theorem 7.7. Let (Q,n,g,h,i) be a mixed quiver setting satisfying Then the algebra of 

invariants ¥[H(Q, n, h)\ G ^ n,B ' 1 ' is generated as V-algebra by the elements ^> D {a t {X^ ■ ■ ■ Xp r )), 
^ D {bpi T (Y u ...,Y s )), where 

1. Pi ■ ■ ■ (3 r ranges over all closed paths in Q D and 1 < t < n^; 

2. (T, (Yi, . . . , Y s )) ranges over all path £l D -tableaux with substitutions of a weight w such that 

a ) tf 9v ^ {GL, O, Sp} for some v £ Qo, then = w v = 0; 

b) if 9 V = f or some v £ Qo, then =0 or w v = 0; 

c ) */ Qv = SO for some ti e Qo, then w v < 1 and i(v) = v. 

Remark 7.8. Semi-invariants of mixed representations of quivers were found by Lopatin and 
Zubkov in [30j . This result was generalized for an arbitrary quiver setting by Lopatin in [28j , where 
Theorem 17.71 was proven. In particular, Theorem 17.71 implies the description of semi- invariants of 
supermixed representations. 

Acknowledgements. This paper was supported by RFFI 07-01-00392. 
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